Cascaded Convolutional Neural Network Research Articles

Automatic Music Recommendation System Algorithm Using Facial Expression Based on Machine Learning

Abstract: Music plays a significant role in improving and elevating one’s mood as it is one of the important source of entertainment and inspiration to move forward. Recent studies have shown that humans respond as well as react to music in a very positive manner and that music has a high impact on human’s brain activity. Now a days, people often prefer to listen to music based on their moods and interests. This work focuses on a system that suggests songs to the users, based on their state of mind. In this system, computer vision components are used to determine the user’s emotion through facial expressions. Once the emotion is recognized, the system suggests a song for that emotion, saving a lot of time for a user over selecting and playing songs manually. Conventional method of playing music depending upon the mood of a person requires human interaction. Migrating to the computer vision technology will enable automation of such system. To achieve this goal, an algorithm is used to classify the human expressions and play a music track as according to the present emotion detected. It reduces the effort and time required in manually searching a song from the list based on the present state of mind of a person. The expressions of a person are detected by extracting the facial features using the Haar Cascade algorithm and CNN Algorithm. An inbuilt camera is used to capture the facial expressions of a person which reduces the designing cost of the system as compared to other methods.

Attendance System Face Recognition using Convolutional Neural Network (CNN)

This article discusses the development of technology in various fields, with a focus on the implementation of digital technology and machine learning. Digitalization has influenced various aspects of life, including education and tourism. Machine learning, particularly convolutional neural networks (CNNs) and deep learning, play an important role in these advancements, with applications extending from biology to healthcare. Face recognition technology, as part of biometrics, is highlighted in this article, used in various contexts such as security and enterprise management. This research implements CNN and Haar Cascade Classifier methods to build a face recognition system in the context of library attendance. With the tests conducted, the system achieved 95% accuracy, showing a good ability to detect faces in various conditions. In conclusion, the CNN algorithm is able to produce an effective face recognition system for use in library attendance systems, with reliable performance and high accuracy.

Movie and Music Recommendation System based on Facial Expressions

Abstract: This paper depicts a novel approach of recommendation system created to extract user’s facial expressions and analyze it using Convolutional Neural Network. CNN is a type of artificial neural network which primarily is used for image recognition and processing by recognizing patterns in images. Initially, an inbuilt webcam will operate to detect user’s facial expression which reduces system’s cost compared to alternative methods. The main concern in existing recommendation systems is manual sorting, so to avoid it, this model is proposed. ‘Movie and Music Recommendation System based on Facial expressions’ provides a way to automatically play the music and movie without spending much time browsing it. The most common problem we all face is, whenever we hop onto third party movies and music platforms, we find tons of options and get stuck into the confusion of what to watch in terms of movies and spend half of our time surfing the options and then we decide to watch something specific. Computer vision is used to understand people in images and videos. This research project introduces an ingenious recommendation system that capitalizes on facial expressions in real time, customize movie and music suggestions through emotion analysis. Haar Cascade algorithm and CNN targets to detect facial feature detection. The database of movies is routed to IMDB site and songs to Spotify API. Training of dataset is done on FER 2013 sample images. It has an input layer, convolutional layer, dense layer and output layer. After the emotions are detected, mood classification is done (for song: tone, valence, timber; for movie: happy, surprise, sad, neutral, anger.) The model is displayed on a website created using Html and CSS with Flask as interface. This system yields better performance results. This recommendation system is exhibited on a website which is user friendly created with HTML and CSS with Flask.

Automatic lung and colon cancer detection using enhanced cascade convolution neural network

Automatic lung and colon cancer detection using enhanced cascade convolution neural network

Space–Time Adaptive Processing Using Deep Neural Network-Based Shrinkage Algorithm Under Small Training Samples

This correspondence discusses a space-time adaptive processing (STAP) method using deep neural network (DNN) and shrinkage algorithm (SA) under small training samples. A DNN-based SA is developed to estimate the clutter-plus-noise covariance matrix (CNCM) from the sample covariance matrix. This technique establishes a learning-based nonlinear mapping between the sample eigenvalues and the optimal shrunk eigenvalues and thus provides an accurate CNCM estimation. Specifically, a DNN consisting of cascaded convolutional neural network (CNN) and fully connected network (FCN) is designed, where the CNN performs the feature extraction and the FCN derives the estimation of the shrunk eigenvalues. To improve the CNN performance, a deep residual learning strategy is introduced into CNN. Due to the lack of labeled data, an airborne radar system is simulated for the construction of dataset consisting of sample eigenvalues and optimal shrunk eigenvalues. Numerical results demonstrate that the proposed DNN-based SA STAP has superior performance in comparison with the state-of-the-art SA STAP methods.

A cascaded convolutional neural networks for stroke detection imaging

In recent years, electrical impedance tomography has widely been used in stroke detection. To improve the prediction accuracy and anti-noise ability of the system, the inverse problem of electrical impedance tomography needs to be solved, for which cascade convolutional neural networks are used. The proposed network is divided into two parts so that the advantages can be compounded when parts of a network are cascaded together. To get high-resolution imaging, an optimized network based on encoding and decoding is designed in the first part. The second part is composed of a residual module, which is used to extract the characteristics of voltage information and ensure that no information is lost. The anti-noise performance of the network is better than other networks. In physical experiments, it is also proved that the algorithm can roughly restore the location of the object in the field.

Classification of Hyperspectral and LiDAR Data Using Multi-Modal Transformer Cascaded Fusion Net

With the continuous development of surface observation methods and technologies, we can acquire multiple sources of data more effectively in the same geographic area. The quality and availability of these data have also significantly improved. Consequently, how to better utilize multi-source data to represent ground information has become an important research question in the field of geoscience. In this paper, a novel model called multi-modal transformer cascaded fusion net (MMTCFN) is proposed for fusion and classification of multi-modal remote sensing data, Hyperspectral Imagery (HSI) and LiDAR data. Feature fusion and feature extraction are the two stages of the model. First, in the feature extraction stage, a three-branch cascaded Convolutional Neural Network (CNN) framework is employed to fully leverage the advantages of convolutional operators in extracting shallow-level local features. Based on this, we generated multi-modal long-range integrated deep features utilizing the transformer-based vectorized pixel group transformer (VPGT) module during the feature fusion stage. In the VPGT block, we designed a vectorized pixel group embedding that preserves the global features extracted from the three branches in a non-overlapping multi-space manner. Moreover, we introduce the DropKey mechanism into the multi-head self-attention (MHSA) to alleviate overfitting caused by insufficient training samples. Finally, we employ a probabilistic decision fusion strategy to integrate multiple class estimations, assigning a specific category to each pixel. This model was experimented on three HSI-LiDAR datasets with balanced and unbalanced training samples. The proposed model outperforms the other seven SOTA approaches in terms of OA performance, proving the superiority of MMTCFN for the HSI-LiDAR classification task.

Edge-Oriented Social Distance Monitoring System Based on MTCNN

Social distance monitoring (SDM) systems are vital in fighting the spread of the coronavirus (COVID-19). Existing SDM systems employ bounding box-method, which imposes inaccurate distance estimation due to the high variance in its output coordinates. To solve this problem, an SDM system based on multitask cascaded convolutional neural networks (MTCNN) is proposed. Instead of using bounding box coordinates, face detection and facial landmarks localization of MTCNN is used to provide fixed coordinates and increase the distance estimation accuracy of SDM. However, while the accuracy issue is solved by using MTCNN, the SDM system suffer from large computational requirements due to the cascaded networks added on top of the distance estimation process. To deal with this challenge, a constrained optimization technique is employed to each stage of MTCNN with the goal of reducing its hardware requirements while keeping the same reliability as the original implementation. Experimental results show that the SDM system based on the optimized MTCNN achieves higher accuracy performance with reduced computational requirements as compared to conventional SDM systems. This allows the proposed SDM system using optimized MTCNN to be deployed efficiently on edge devices.

Dualistic cascade convolutional neural network dedicated to fully PolSAR image ship detection

Influenced by the imaging mechanism, the occurrence of interference clutter in synthetic aperture radar (SAR) renders the identification of false alarms using detectors challenging. Polarimetric SAR has the potential to improve ship detection performance owing to its distinctive polarization characteristics. The present study proposes a dualistic cascade convolutional neural network (DCCNN) algorithm driven by polarization characteristics for ship detection with fully PolSAR data. First, the new characterizations of fully PolSAR data—Optimized SPAN (OSPAN) and 6-D polarization vector (P6), were mined and defined based on the polarization coherence matrix to introduce more intact information of targets. Then, a backbone feature extraction network with parallel dualistic cascade architecture, basic geometric feature extraction network (BGFENet), and polarization feature enhancement network (PFENet) was specifically constructed, which provided the comprehensive feature representation of targets via feature fusion. Finally, the classification and regression tasks were accomplished in the fully convolutional detection subnetwork relying on extracted multi-scale fusion feature maps, while focusing on target location regression, leading to a decrease in the cost of detection efficiency caused by redundant PFENet. In addition, the corresponding training strategy according to the special architecture of DCCNN was designed to overcome the problem of labeled fully PolSAR data insufficiency and migrate refined geometric knowledge of targets in the single-polarization SAR amplitude image. Experimental results on the established fully polarized SAR dataset show that DCCNN outperforms the competitive CNN-based target detection methods by at least 6.79% in terms of average precision. Moreover, experimental results on the typical large scenes show that DCCNN is at least 0.88% higher than the well-known conventional methods in terms of F1 score.

A Learning-Based Methodology for Microwave Passive Component Design

Microwave passive component design is of particular interest to radio frequency (RF) scholars and engineers. Although a plethora of studies have been carried out over multiple decades, designing high-frequency structures that offer high performance still heavily relies on heuristic methods and even rules of thumb. Thus, the process is often inefficient, and outcomes are not guaranteed. This article proposes a novel cascaded convolutional neural network (CNN) model to speed up the design process of planar microwave passive components. Given target behavior specifications, our proposed neural network model can quickly and accurately suggest proper component structures for single or multiple frequency bands. The feasibility and reliability of our model are validated here by both electromagnetic (EM) simulation and a fully instrumented implementation. The experimental results demonstrate that the proposed model can design planar passive components, including two-port matching networks and three-port power dividers. Moreover, our model provides passive component topologies that are fundamentally different from canonical number-limited templates and, therefore, yields novel architectures for passive microwave components. It also facilitates rapid passive components design flow for targeted electrical behavior within a limited board area. The proposed cascaded CNN model and the associated methodologies in this article are generic and, thus, can be easily extended to the design of any symmetrical planar microwave passive components.

Non-invasive, intelligent, and continuous monitoring of human blood glucose with UWB dual-antenna and cascade CNN

Finger-prick blood collection process has become unrealistic for a long-term and frequent blood glucose detection. Hence, an appropriate non-invasive detection system is highly desirable to effectively address this concern. A non-invasive and intelligent dual-sensing system is forwarded in this paper. The feasibility of the proposed system has been verified using glucose solution, animal serum, and human trials. In the in vivo experiments, the detection signal exhibited a high correlation (r = 0.96) with blood glucose levels. An improved cascade convolution neural network is suggested to accurately predict the BGL. For the estimation results of BGL, the root mean squared error of 7.3217 mg dl−1 and a mean absolute relative difference of 4.7209% are achieved. The estimated results also fell by 100% in the clinically acceptable zones of the Clarke error grid analysis, indicating that the proposed system could potentially be used for clinical measurements.

MARC-Net: Terrain Classification in Parallel Network Architectures Containing Multiple Attention Mechanisms and Multi-Scale Residual Cascades

To address the problem that traditional deep learning algorithms cannot fully utilize the correlation properties between spectral sequence information and the feature differences between different spectra, this paper proposes a parallel network architecture land-use classification based on a combined multi-head attention mechanism and multiscale residual cascade called MARC-Net. This parallel framework is firstly implemented by deeply mining the features generated by grouped spectral embedding for information among spectral features by adding a multi-head attention mechanism, which allows semantic features to have expressions from more subspaces while fully considering all spatial location interrelationships. Secondly, a multiscale residual cascade CNN (convolutional neural network) is designed to fully utilize the fused feature information at different scales, thereby improving the network’s ability to represent different levels of information. Lastly, the features obtained by the multi-head attention mechanism are fused with those obtained by the CNN, and the merged resultant features are downgraded through the fully connected layer to obtain the classification results and achieve pixel-level multispectral image classification. The findings show that the algorithm proposed in this paper has an aggregate precision of 97.22%, compared to that of the Vision Transformer (ViT) with 95.08%; its performance on the Sentinel-2 dataset shows a huge improvement. Moreover, this article mainly focuses on the change rate of forest land in the study area. The Forest land area was 125.1143 km2 in 2017, 105.6089 km2 in 2019, and 76.3699 km2 in 2021, with an increase of 15.59%, an decrease of 0.97%, and increase of 14.76% in 2017–2019, 2019–2021 and 2017–2021, respectively.

Synthetic aperture radar images denoising based on multi-scale attention cascade convolutional neural network

Synthetic aperture radar (SAR) images are often affected by speckle noise, which can hinder accurate interpretation and subsequent use of the images in applications such as target detection and segmentation. To address this issue, we propose a denoising algorithm based on a multi-scale attention cascade convolutional neural network (MSAC-Net). Our algorithm employs multi-scale asymmetric convolution to extract image features and an attention mechanism to integrate these features. Additionally, we designed a multi-layer deep cascade convolutional network to enhance the generalization ability of the model features. Experimental results show that our proposed MSAD-Net model significantly outperforms state-of-the-art SAR image denoising algorithms. Specifically, it achieves a significant improvement in peak signal-to-noise ratio, with an increase of about 0.81–13.97 dB, and structural similarity index measure, with an increase of about 0.01–0.14. Overall, our study presents a novel denoising algorithm for SAR images that greatly improves the accuracy of subsequent image applications.

Proposed Face Detection Classification Model Based on Amazon Web Services Cloud (AWS)

One of the most important features of the Amazon Web Services (AWS) cloud is that the program can be run and accessed from any location. You can access and monitor the result of the program from any location, saving many images and allowing for faster computation. This work proposes a face detection classification model based on AWS cloud aiming to classify the faces into two classes: a non-permission class, and a permission class, by training the real data set collected from our cameras. The proposed Convolutional Neural Network (CNN) cloud-based system was used to share computational resources for Artificial Neural Networks (ANN) to reduce redundant computation. The test system uses Internet of Things (IoT) services through our cameras system to capture the images and upload them to the Amazon Simple Storage Service (AWS S3) cloud. Then two detectors were running, Haar cascade and multitask cascaded convolutional neural networks (MTCNN), at the Amazon Elastic Compute (AWS EC2) cloud, after that the output results of these two detectors are compared using accuracy and execution time. Then the classified non-permission images are uploaded to the AWS S3 cloud. The validation accuracy of the offline augmentation face detection classification model reached 98.81%, and the loss and mean square error were decreased to 0.0176 and 0.0064, respectively. The execution time of all AWS cloud systems for one image when using Haar cascade and MTCNN detectors reached three and seven seconds, respectively.

Automated diagnosis of flatfoot using cascaded convolutional neural network for angle measurements in weight-bearing lateral radiographs.

Diagnosis of flatfoot using a radiograph is subject to intra- and inter-observer variabilities. Here, we developed a cascade convolutional neural network (CNN)-based deep learning model (DLM) for an automated angle measurement for flatfoot diagnosis using landmark detection. We used 1200 weight-bearing lateral foot radiographs from young adult Korean males for the model development. An experienced orthopedic surgeon identified 22 radiographic landmarks and measured three angles for flatfoot diagnosis that served as the ground truth (GT). Another orthopedic surgeon (OS) and a general physician (GP) independently identified the landmarks of the test dataset and measured the angles using the same method. External validation was performed using 100 and 17 radiographs acquired from a tertiary referral center and a public database, respectively. The DLM showed smaller absolute average errors from the GT for the three angle measurements for flatfoot diagnosis compared with both human observers. Under the guidance of the DLM, the average errors of observers OS and GP decreased from 2.35° ± 3.01° to 1.55° ± 2.09° and from 1.99° ± 2.76° to 1.56° ± 2.19°, respectively (both p < 0.001). The total measurement time decreased from 195 to 135min in observer OS and from 205 to 155min in observer GP. The absolute average errors of the DLM in the external validation sets were similar or superior to those of human observers in the original test dataset. Our CNN model had significantly better accuracy and reliability than human observers in diagnosing flatfoot, and notably improved the accuracy and reliability of human observers. • Development of deep learning model (DLM) that allows automated angle measurements for landmark detection based on 1200 weight-bearing lateral radiographs for diagnosing flatfoot. • Our DLM showed smaller absolute average errors for flatfoot diagnosis compared with two human observers. • Under the guidance of the model, the average errors of two human observers decreased and total measurement time also decreased from 195 to 135min and from 205 to 155min.

ME-CCNN: Multi-encoded images and a cascade convolutional neural network for breast tumor segmentation and recognition

ME-CCNN: Multi-encoded images and a cascade convolutional neural network for breast tumor segmentation and recognition

Landmark tracking in liver US images using cascade convolutional neural networks with long short-term memory

Accurate tracking of anatomic landmarks is critical for motion management in liver radiation therapy. Ultrasound (US) is a safe, low-cost technology that is broadly available and offer real-time imaging capability. This study proposed a deep learning-based tracking method for the US image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from an US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box regression, proposal classification, and landmark segmentation. The LSTM network models the temporal relationship among the successive image frames for bounding box regression and proposal classification. To consolidate the final proposal, a selection method is designed according to the similarities between sequential frames. The proposed method was tested on the liver US tracking datasets used in the medical image computing and computer assisted interventions 2015 challenges, where the landmarks were annotated by three experienced observers to obtain their mean positions. Five-fold cross validation on the 24 given US sequences with ground truths shows that the mean tracking error for all landmarks is 0.65 ± 0.56 mm, and the errors of all landmarks are within 2 mm. We further tested the proposed model on 69 landmarks from the testing dataset that have the similar image pattern with the training pattern, resulting in a mean tracking error of 0.94 ± 0.83 mm. The proposed deep-learning model was implemented on a graphics processing unit (GPU), tracking 47–81 frames s−1. Our experimental results have demonstrated the feasibility and accuracy of our proposed method in tracking liver anatomic landmarks using US images, providing a potential solution for real-time liver tracking for active motion management during radiation therapy.

Numerical and Clinical Evaluation of the Robustness of Open-source Networks for Parallel MR Imaging Reconstruction.

Deep neural networks (DNNs) for MRI reconstruction often require large datasets for training. Still, in clinical settings, the domains of datasets are diverse, and how robust DNNs are to domain differences between training and testing datasets has been an open question. Here, we numerically and clinically evaluate the generalization of the reconstruction networks across various domains under clinically practical conditions and provide practical guidance on what points to consider when selecting models for clinical application. We compare the reconstruction performance between four network models: U-Net, the deep cascade of convolutional neural networks (DC-CNNs), Hybrid Cascade, and variational network (VarNet). We used the public multicoil dataset fastMRI for training and testing and performed a single-domain test, where the domains of the dataset used for training and testing were the same, and cross-domain tests, where the source and target domains were different. We conducted a single-domain test (Experiment 1) and cross-domain tests (Experiments 2-4), focusing on six factors (the number of images, sampling pattern, acceleration factor, noise level, contrast, and anatomical structure) both numerically and clinically. U-Net had lower performance than the three model-based networks and was less robust to domain shifts between training and testing datasets. VarNet had the highest performance and robustness among the three model-based networks, followed by Hybrid Cascade and DC-CNN. Especially, VarNet showed high performance even with a limited number of training images (200 images/10 cases). U-Net was more robust to domain shifts concerning noise level than the other model-based networks. Hybrid Cascade showed slightly better performance and robustness than DC-CNN, except for robustness to noise-level domain shifts. The results of the clinical evaluations generally agreed with the results of the quantitative metrics. In this study, we numerically and clinically evaluated the robustness of the publicly available networks using the multicoil data. Therefore, this study provided practical guidance for clinical applications.

ConvTransNet: A CNN–Transformer Network for Change Detection With Multiscale Global–Local Representations

Change detection (CD) in optical remote sensing images has significantly benefited from the development of deep convolutional neural networks (CNNs) due to their strong capability of local modeling in bi-temporal images. In addition, the recent rise of transformer modules leads to the improvement of global feature extraction of bi-temporal remote sensing images. Note that the existing simple cascade of deep CNNs and transformer modules shows limited CD performance on small changed areas due to deficiencies of multi-scale information therein. To address the aforementioned issue, we propose a new CNN-transformer network (ConvTransNet) with multi-scale framework to better exploit global-local information in optical remote sensing images. In our ConvTransNet, we propose the parallel-branch ConvTrans block as the basic component to generate global-local features, i.e., adaptively integrates the global features summarized by a transformer-based branch and the local features extracted by a convolution-based branch, providing better identifiability between changed areas and unchanged areas. By fusing multiple global-local features with different scales, our ConvTransNet improves the robustness of the CD performance on changed areas with different sizes, especially small changed areas. Experiments on two public change detection datasets of optical remote sensing images, i.e., LEVIR-CD and CDD, demonstrate that our ConvTransNet achieves enhanced CD performance than the other commonly used methods.

Multi illumination color constancy based on multi-scale supervision and single-scale estimation cascade convolution neural network.

Color constancy methods are generally based on a simplifying assumption that the spectral distribution of a light source is uniform across scenes. However, in reality, this assumption is often violated because of the presence of multiple light sources, that is, more than two illuminations. In this paper, we propose a unique cascade network of deep multi-scale supervision and single-scale estimation (CN-DMS4) to estimate multi-illumination. The network parameters are supervised and learned from coarse to fine in the training process and estimate only the final thinnest level illumination map in the illumination estimation process. Furthermore, to reduce the influence of the color channel on the Euclidean distance or the pixel-level angle error, a new loss function with a channel penalty term is designed to optimize the network parameters. Extensive experiments are conducted on single and multi-illumination benchmark datasets. In comparison with previous multi-illumination estimation methods, our proposed method displays a partial improvement in terms of quantitative data and visual effect, which provides the future research direction in end-to-end multi-illumination estimation.